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Abstract 

The main aim of this study is to determine whether a significant difference exits among data collection with 
printed materials and with smart phones. The research was conducted with 282 teacher candidates who take 
pedagogical formation training. Three data collection tools were used throughout the study. As the results of the 
current research, no significant differences were obtained among attitude scores towards teaching profession 
with regard to common effect of application method and faculty variables. Flowever, significant differences in 
participant opinions were determined among different application methods (smartphone and printed) of 
Metacognitive Thinking Skills and Information Literacy Scales. In addition, no significant difference was found 
among opinions towards Metacognitive Thinking Skills and Information Literacy Scales with regard to common 
effect of application method and faculty variables. Based on these research findings, it is not recommended to 
distribute survey items containing technical information or some complex expressions via smartphones. 
Relations among the questionnaire content and the department of education confirm this situation. 

Keywords: smartphone. Online survey. Information Literacy, Metacognition Skills, Attitude towards teaching 
profession 

1. Introduction 

Technology is able to change human behavior through innovations it presents. Without going through scientific inquiry, 
it is not possible for us to know what changing human behaviors really are and how they can be explained. Change at 
such rapid rate and sectorial differences, is causing difficulties in explaining and managing them. Throughout this study, 
this situation which appears in many sectors is handled through its educational aspect. 

Survey is among the research methods that researchers apply to in social sciences. With the effect of quantitative 
paradigm, researchers have been referring to a number of ways within survey method to maintain validity and reliability 
of data collection tools and to reach large samples. Advances in technology have opened the gate to online applications 
of data collection tools to achieve these goals. Over time, this opportunity has become quite widespread, but possible 
limitations of this new data collection method have not been an issue thoroughly considered. 

Current situation of technology based learning is an issue of interest for a number of researchers. Remarkable research 
studies can be seen in literature on human behaviors and impact of technology. For example, it was observed that the 
internet has begun to be seen as the primary source for learning (Tuncer, Yilmaz and Tan, 2011; Tuncer and Kaysi, 2011) 
and also with the prevailing internet technology, disposition of getting information through books and libraries is 
gradually weakening whereas we approach to a point where learning in a virtual environment, called screen reading, is 
being preferred (Tuncer and Bahadrr, 2014). But according to a number of researches (More, Guy and Elobaid, 2007; 
Alshaali and Varshney, 2005; Annand, 2008; Weeks, 2002; Spencer, 2006; Vernon, 2006) learners prefer printed 
materials instead of screen reading. Besides, according to Stoop, Kreutzer and Kircz (2013) e-readers display such 
behaviors as not being able to take notes with ease, slow reading, postponing examinations and printing out e-books. 
Johnson (2000) explained the reason of preferring printed material for reading as successful readers get bored of simple 
texts and weak readers abandon reading texts which are not fluent. 

Reading is important for learning, but reading comprehension is important as well. CjJelik (2006), states that reading is 
composed of various functions of eye, speech organs and cognition such as perceiving what is seen, comprehending 
what is perceived, stating what is comprehended and constructing in mind. Studies on reading effectiveness in learning 
differ in terms of organ functions. For instance, reading is slower in oral reading due to attention and time loss. 
Meantime, semantic loss can also occur (MNE (Ministry National Education), 2011). Compared to silent reading, oral 
reading is more beneficial in learning short texts because it addresses both eyes and ears (Ayta§, 2005). During silent 
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reading, words and sentences are followed by eye without vocalizing. As soon as the eye perceives, it transmits text to 
memory to be processed. In this reading type (silent reading), vocal cords, sound waves, tongue and ear stay out of the 
reading process (Giine§, 2007). In silent reading, connection of vision-comprehension is needed. It is seen as a faster 
type of reading compared to oral reading and is considered important due to acuteness it provides (Muftuoglu and K 09 , 
1998: 64). According to (j’elik (2006), the brain comprehends what is read faster for what eyes see keeps it busy enough 
and it cannot find any other endeavor. 

Joy Bolter pointed computers as the fourth large document medium after papyrus plant, which ancient Egyptians made 
sails, fabric, straw and writing paper out of its stem, medieval books and printed books (O’Hara and Sellen, 1997). 
According to Dyson (2004) besides rapid development of web technology, also spreading word processers used to 
create documents increased number of screen-read documents. Executing most of the studies in the field of education 
through computers, projecting documents such as e-books, articles and online journals onto the screens using 
computer-type devices show that screen-reading has become a necessity at schools as well. According to Maden (2012), 
screen-reading is a type of reading which an individual encounter frequently but uses unwittingly. Anameri 9 and 
Rukanci (2003) stated that owning a printed book is a privilege, touching it and sensing its smell arouse different 
feelings. On the contrary, e-books have some conveniences such as lower prices compared with printed books, easier 
portability and dissemination possibility over the internet. Ylldmm and others ( 2011 ) suggest that screen-reading may 
be advantageous to reading from printed materials due to adjustable screen size and resolution. Reading would be 
comfortable if screen page is A4 or A5 size, emphasized Mallet (2010). Furthermore, while large text screen is 
necessary to be able to see the whole page, it is not found favorable as it can create a physical weight. 

May these positive and negative findings towards on-screen text reading appear during data collection by online surveys 
which researchers frequently apply? Researchers may prefer online data collection for a number of reasons such as 
reaching different and larger samples, more economical data collection and avoiding unanswered questions (where 
answering a question is obligatory). As a matter of fact, in the era of the internet, Strachota, Schmidt and Conceicao 
(2005) see online surveys as an efficient way of data collection. Schmidt, Strachota and Conceipao (2006) and also Rose 
and Bogue (2006) see online questionnaire distribution advantageous from a number of point of views such as saving 
time, reducing data entry error, increasing product response rate and reducing cost. The disadvantages of the online 
survey in this research include the need for specific computer skills such as use of pop-up menus and scrolling, the need 
for user-friendly software to maintain careful reading of instructions and paying attention to detail. Gunn (2002) 
believes that web page design skills and computer programming expertise play an important role in web-based 
questionnaire design, unlike other types of questionnaires. Timmerman (2002) emphasizes that online questionnaires 
should be investigated as to whether the target audience is appropriate for the research topic. In a study they conducted, 
Rosa, Bressan and Toledo (2012) concluded that online questionnaires might be used, but they had certain limitations in 
terms of tools. According to Rose and Bogue (2006), the most important issue to be considered when evaluating on-line 
questionnaires is the opportunity to access the internet and quality of the access. Fleming and Bowden (2009) pointed 
out that sampling biases in web-based surveys may cause problems in terms of research findings. Similarly, Bushanan 
and Hvizdak (2009) pointed the dimensions such as security, privacy, sampling, consent, design and spamming in 
web-based surveys. Due to the current technology, the management of online surveys has resulted in increased time 
efficiency by both distributing and retrieving survey results online. However, this does not change the fact that the main 
criterion of the measurement is to reach valid data. Despite the fact that, online data collection is preferred in many 
studies, researchers have hardly ever questioned this mandatory criterion. There are a number of conducted studies 
closely related to the research topic (Fleming and Bowden , 2009; Nulty, 2008; Strachota, Schmidt and Conceicao, 2005; 
Schmidt, Strachota and Conceipao, 2006; Rose and Bogue, 2006; Morris, Woo and Cho, 2003; Szolnoki and Hoffman, 
2013; Wiersme, 2013). However, findings may be related to data collection tools utilized. For this reason, findings 
obtained in the study were evaluated according to primarily to the method utilized. Secondly, multiple data collection 
tools were used in the sense that the findings could be derived from data collection tools. 

2. Method 

The design of this study can said to be a survey model. As Karasar (2009) puts it, it is aimed to describe past or present 
situations in the survey model. The main aim of this study is to determine whether a significant difference exits among 
data collection with printed materials and with smart phones. Studies on comparison of printed and internet based 
applications (smartphones) of data collection tools are seen in the related literature. However, no studies which are free 
from scale effect and are on comparison of printed and smart phone survey scores have been encountered. With the use 
of smart phones more functionally in every environment, it can be stated that smart phones will start to replace 
computers for accessing the internet. Therefore, it may be considered that the current research topic contains a specific 
problem situation. 

The research was conducted with 282 teacher candidates who take pedagogical formation training at Firat University 
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Faculty of Education. 29 teacher candidates who did not accept to participate and 33 candidates who did not own 
smartphones were excluded from the actual sample size of 344 people. Thus, it was ensured that the research was 
conducted on a voluntary group having smart phone skills. The distribution of teacher candidates in respect to their 
faculties is as; Sport Sciences (N=59), Humanities and Social Sciences (N=62), Economics and Administrative Sciences 
(N=60), Science (N=50) and Theology (N=51). Virtual survey group consists of 155 while printed survey group 
consists of 127 teacher candidates. Groups were assigned randomly. 

Independent samples t test, coefficient of variation, effect size and two-way analysis of variance were used to evaluate 
research data. A coefficient of variation smaller than 20 is interpreted as a homogenous distribution while a value 
between 20 and 25 is interpreted as a normal distribution and a value larger than 25 is interpreted as a heterogeneous 
distribution (Karaca, 2008: 264). Intervals stated by Green and Salkind (1997; cited by Buyiikbztiirk, £okluk and Koklii, 
2012:189) and Cohen’s (1988) effect sizes (> 0.5: strong, > 0.3: medium and > .01 weak) were taken into consideration 
together for the interpretation of effect size. 

3. Data Collection Tools 

Three data collection tools were used throughout the study. The Information Literacy Self-Efficacy Scale (ILSE) was 
developed by Kurbanoglu, Akkoyunlu and Umay (2006) and adapted to Turkish by Tuncer (2013). The tool is a 7-point 
Likert type scale which enables “Almost Every Time True”, “Generally True”, “Often True”, “Sometimes True”, 
“Rarely True”, “Generally not True”, “Almost Never Tine” responses. Scoring was done as: 7 for “Almost Every Time 
True”, 6 for “Generally True”, 5 for “Often True”, 4 for “Sometimes True”, 3 for “Rarely True”, 2 for “Generally not 
True” and 1 for “Almost Never True”. The second data collection tool of the study, Metacognitive Thinking Skills Scale 
(MCTS) was developed by Tuncer and Kaysi (2013). This scale consists of 18 items under 4 dimensions. These 
dimensions are: thinking skills, reflective thinking skills for problem solving, decision making skills and alternative 
assessment skills. Four dimensions of the scale explain 56.579% of total variance. Five-point Likert type scales are 
scored as 5 for “Completely Agree”, 4 for “Agree”, 3 for “Undecided”, 2 for “Disagree” and 1 for “Completely 
Disagree”. The third data collection tool of the study is Attitude towards Teaching Profession (ATTP) Scale which has 
35 items, was developed by Qetin (2006). Each student who responded, scored according to a five-point Likert type 
scale: 5 for “Strongly Agree”, 4 for “Agree”, 3 for “Undecided”, 2 for “Disagree” and 1 for “Never Agree”. On the 
other hand, each student scored the items with a negative stem as: 1 for “Strongly Agree”, 2 for “Agree”, 3 for 
“Undecided”, 4 for “Disagree” and 5 for “Never Agree”. 

Information Literacy Scale consists of 4 sub-dimensions (library literacy, information literacy, bibliography literacy and 
scientific research literacy). While MCST Scale consists of 4 sub-dimensions (thinking, problem solving, decision 
making and alternative evaluation). Attitudes towards Teaching Profession Scale (ATTP) consists of 3 sub-dimensions 
(love, value, harmony). Reliability coefficients were found as .874 for MCST Scale, .935 for Information Literacy Scale 
and .914 for Attitudes towards Teaching Profession Scale. 

4. Findings 

Comparisons of findings towards scales filled in online and printed, in terms of sub-dimensions, distribution of the 
answers and effects of the application method on the responses, are presented in Table 1. 

It was determined that opinions regarding Attitude towards Teaching Profession Scale did not differ significantly (p>.05) 
according to the application method. Besides, coefficient of variability showed homogenous distributions of the answers 
in terms of love, value and harmony sub-dimensions and the whole scale. Significant differences (p<.05) among all 
sub-dimensions of Metacognitive Thinking and Information Literacy scales were also observed according to the 
application method. When the scale dimensions with significant differences among the opinions were evaluated in terms 
of effect size, it was determined that the application method was strongly effective on thinking and problem solving 
sub-dimensions and on the whole scale, but moderately effective on alternative assessment sub-dimension of MCST 
scale and also strongly effective on all sub-dimensions and the whole of Library Literacy scale. 
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Table 1. Analyses on comparison of the responses according to the application method and distribution of the answers 


Scale 

Dimension 

Application X 

SD 

V 

t 

P 

Distribution 

Ef. Size 


Love 

Smartphone 

3,90 

,71 

18,20 

-1,230 

,220 

homogeneous 



Printed 

4,00 

,72 

18,00 

homogeneous 



Value 

Smartphone 

4,56 

,48 

10,52 

,736 

,998 

homogeneous 


ATTPS 

Printed 

4,56 

,47 

10,30 

homogeneous 


Harmony 

Smartphone 

3,65 

,47 

12,87 

1,923 

,055 

homogeneous 



Printed 

3,52 

,70 

19,88 

homogeneous 



Whole 

Smartphone 

4,01 

,55 

13,71 

-1,733 

,464 

homogeneous 



Printed 

4,06 

,57 

14,03 

homogeneous 



Thinking 

Smartphone 

4,27 

,40 

9,37 

9,031 

,000* 

homogeneous 

,231 


Printed 

3,68 

,68 

18,48 

homogeneous 


Problem Solv. 

Smartphone 

3,98 

,52 

13,07 

3,869 

,000* 

homogeneous 

,066 


Printed 

3,68 

,77 

20,92 

Normal Dist. 

MCTS 

Decision 

Smartphone 

4,18 

,54 

12,92 

3,363 

,001* 

homogeneous 

,096 

Making 

Printed 

3,92 

,76 

19,39 

homogeneous 


Alternative 

Smartphone 

3,95 

,52 

13,16 

2,969 

,003* 

homogeneous 

,030 


Evaluation 

Printed 

3,71 

,80 

21,56 

Normal Dist. 


Whole 

Smartphone 

4,11 

,38 

9,25 

6,260 

,000* 

homogeneous 

,144 


Printed 

3,74 

,60 

16,04 

homogeneous 


Library 

Smartphone 

4,66 

1,36 29,18 

4,677 

,000* 

heterogeneous 

,086 


Printed 

3,83 

1,62 42,30 

heterogeneous 


Infonnation 

Smartphone 

5,33 

,95 

17,82 

2,746 

,006* 

homogeneous 

,042 


Printed 

5,00 

1,02 20,40 

Normal Dist. 

ILSE 

Bibliography 

Smartphone 

4,66 

1,44 30,90 

4,329 

,000* 

heterogeneous 

,079 

Printed 

3,93 

1,46 37,15 

heterogeneous 


Sci. Research 

Smartphone 

5,04 

1,14 22,62 

2,998 

* 

co 

o 

o 

Normal Dist. 

,055 


Printed 

4,64 

1,12 24,14 

Normal Dist. 


Whole 

Smartphone 

4,98 

1,00 20,08 

4,429 

* 

o 

o 

o 

Normal Dist. 

,089 


Printed 

4,45 

1,01 22,701 

Normal Dist. 

V: coefficient of variation (%), Smartphone (N=157), Printed (N=127) 


During the research, two-way analysis of variance was performed with respect to the application method and faculty. The 
results of two-way analysis of variance regarding the Attitude Toward Teaching Profession Scale are given in Table 2. 


Table 2. ANOVA analysis of Attitude Toward Teaching Profession Scale with respect to the application method and 
faculty 


Source 

Sum of Squares 

df 

Mean Squares 

F 

P 

Application 

,464 

1 

,464 

1,476 

,226 

Faculty 

2,125 

4 

,531 

1,689 

,153 

Application x Faculty 

1,319 

4 

,330 

1,409 

,382 

Error 

21,676 

272 

,080 



Total 

2917,768 

282 





According to the table, no significant difference (p>.05) was found among scores of attitudes towards teaching 
profession in terms of common effect of application method and faculty. A line graph displaying this dimension is given 


in Figure 1. 


Estimated Marginal Means of ATTP 



Faculty 

- Sport Sciences 
Human Sceinces 
iEcon. and Man. Sceices 

— Science 
Religion 


Figure 1. Method and Faculty Axial Line Graph of Attitudes towards Teaching Profession Scale 
When an evaluation is made in terms of average scores of the application method, printed survey averages were found 
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lower than online survey averages for the faculties of Sport Sciences, Economics and Administrative Sciences. Averages 
of printed surveys were found higher for the faculties of Science, Theology and Humanities and Social Sciences. 
Average opinion scores for ATTP Scale with regard to application type and faculty are given in Table 3. 

In the virtual application of the Attitudes towards Teaching Profession Scale, the highest average score was found for 
Science Faculty (X = 4.16 ± .40) while the lowest average score was found for Humanities and Social Sciences Faculty 
(X = 3.81 ± .52). Similarly, regarding printed application, the highest average score (X = 4.35 ± .33) was found for 
Science Faculty and the lowest average score was found for Humanities and Social Sciences Faculty (X = 3.95 ± .58). 
When a general evaluation is made, it was seen that virtual survey application (X=4.01 ± .55) produced a lower average 
than printed survey application (X=4.06 ± .57). 

Table 3. Descriptive Statistics Based on Application Method and Faculty (ATTP Scale) 



Faculty 

X 

Std. Deviation 


Sport Sciences 

4,08 

,60 


Human Sciences 

3,81 

,52 

Smartphone 

Econ. and Manag. 

4,15 

,60 

Science 

4,16 

,40 


Religion 

3,88 

,51 


Total 

4,01 

,55 


Sport Sciences 

4,04 

,64 


Human Sciences 

3,95 

,58 

Printed 

Econ. and Manag. 

4,10 

,50 

Science 

4,35 

,33 


Religion 

4,10 

,55 


Total 

4,06 

,57 


Sport Sciences 

4,04 

,64 


Human Sciences 

3,95 

,56 

Total 

Econ. and Manag. 

4,12 

,54 

Science 

4,19 

,39 


Religion 

3,93 

,52 


Total 

4,04 

,56 


In the virtual application of the Attitudes towards Teaching Profession Scale, the highest average score was found for 
Science Faculty (X = 4.16 ± .40) while the lowest average score was found for Humanities and Social Sciences Faculty 
(X = 3.81 ± .52). Similarly, regarding printed application, the highest average score (X = 4.35 ± .33) was found for 
Science Faculty and the lowest average score was found for Humanities and Social Sciences Faculty (X = 3.95 ± .58). 
When a general evaluation is made, it was seen that virtual survey application (X=4.01 ± .55) produced a lower average 
than printed survey application (X=4.06 ± .57). 

Results of two-way analysis of variance of the opinions towards Metacognitive Thinking Skills Scale, the second data 
gathering tool of the research, are summarized in Table 4 in terms of method and faculty variables. 

Table 4. ANOVA analysis of the Opinions towards MCTS Scale with regard to Application Method and Faculty 


Source 

Sum of Squares 

df 

Mean Squares 

F 

P 

Application 

6,084 

1 

6,084 

24,724 

,000 

Faculty 

,801 

4 

,200 

,813 

,518 

Application x Faculty 

,552 

4 

,138 

,561 

,691 

Error 

67,180 

273 

,246 



Total 

4483,659 

283 





According to the table, a significant difference (F (1.273) =24.724, p<.05) among scores of Metacognitive Thinking 
Skills Scale with regard to application method (virtual and printed) was determined. However, no significant difference 
(F (1.273) =24.724, p>.05) was determined among scores of Metacognitive Thinking Skills Scale with regard to 
common effect of application method and faculty. Line graph regarding this situation is like at Figure 2. 
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Estimated Marginal Means of MTS 



App. 

Figure 2. Line Graph of MCTS Scale Based on Method and Faculty 

When an evaluation is performed among average scores of application methods, it is seen that virtual scores are higher 
in all faculties. Average opinion scores for Metacognitive Thinking Skills Scale with regard to application type and 
faculty are given in Table 5. 

Table 5. Descriptive Statistics based on Application Method and Faculty (MCTS Scale) 



Faculty 

X 

Std. Deviation 


Sport Sciences 

4,11 

,29 


Human Sciences 

4,10 

,36 

Smartphone 

Econ. and Manag. 

4,20 

,46 

Science 

4,10 

,29 


Religion 

4,07 

,51 


Total 

4,11 

,38 


Sport Sciences 

3,69 

,62 


Human Sciences 

3,69 

,61 

Printed 

Econ. and Manag. 

3,83 

,63 

Science 

3,60 

,62 


Religion 

3,93 

,34 


Total 

3,74 

,60 


Sport Sciences 

3,91 

,52 


Human Sciences 

3,86 

,56 

Total 

Econ. and Manag. 

3,97 

,59 

Science 

4,02 

,39 


Religion 

4,04 

,47 


Total 

3,95 

,53 


Regarding the virtual application of the Metacognitive Thinking Skills Scale, the highest average point was found for 
Economical and Administrative Sciences Faculty (X = 4.20 ± .46) and the lowest for Theology Faculty (X = 4.07 ± .51) 
± .34) while regarding printed application of the scale, the highest average point was found for Theology Faculty (X = 
3.93 ± .34) and the lowest average point was found for Science Faculty (X = 3.60 ± .62). When a general evaluation 
was made, it was determined that the average of the virtual survey application (X = 4.11 ± .38) was higher than the 
average of the printed survey (X = 3.74 ± .60). 

The third data collection tool of the research is the Information Literacy Scale. Results of the two-way analysis of 
variance for this data collection tool in terms of method and faculty variables are summarized in Table 6. 

Table 6. ANOVA analysis of opinions towards Information Literacy Scale regarding application method and faculty 


Source 

Sum of Squares 

df 

Mean Squares 

F 

P 

Application 

18,438 

1 

18,438 

18,429 

,000 

Faculty 

6,416 

4 

1,604 

1,603 

,174 

Application x Faculty 

7,802 

4 

1,950 

1,949 

,103 

Error 

275,144 

275 

1,001 



Total 

6735,851 

285 





According to the table, a significant difference in participant opinions towards information Literacy Scale was 
determined (F (1.275) =18.429, p<.05) among the virtual and printed applications. However, no significant difference 
(F (1.275) =1.949, p>.05) was found among the opinions towards Information Literacy Scale with regard to common 
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effect of faculty and application method. The line graph displaying this situation is given in Figure 3. 


Estimated Marginal Means of IL 



Figure 3. Line Graph of Information Literacy Scale Based on Method and Faculty 

When an evaluation is made in terms of the average scores according to the method applied, it is seen that the average 
virtual scores are higher in all the facidties. It is also seen that the closest points among the virtual and printed surveys 
of the Information Literacy Scale are for Humanities and Social Sciences Faculty. Regarding the Information Literacy 
Scale, average points of application method with respect to faculties are given in Table 7. 

In the virtual application of the Information Literacy Scale, the highest average point was found for Humanities and 
Social Sciences Faculty (X = 5.38 ± .89) and the lowest average point was found for Sports Sciences Faculty (X = 4.73 
± 1.16) moreover, regarding printed application of the scale, the highest average point was found for Humanities and 
Social Sciences Faculty (4.75 ± 0.93) and the lowest average point was found for Science Faculty (X = 4.12 ± 1.05). 
When a general evaluation was made, it was determined that the average of the virtual survey application (X = 4.99 ± 
1.01) was higher than the average of the printed application (X = 4.46 ± 1.01). 

Table 7. Descriptive Statistics Based on Application Method and Faculty (IL Scale) 



Faculty 

X 

Std. Deviation 


Sport Sciences 

4,73 

1,16 


Human Sciences 

4,81 

,98 

Smartphone 

Econ. and Manag. 

5,38 

,89 

Science 

5,11 

1,02 


Religion 

5,09 

,80 


Total 

4,99 

1,01 


Sport Sciences 

4,25 

1,00 


Human Sciences 

4,75 

,93 

Printed 

Econ. and Manag. 

4,41 

1,10 

Science 

4,12 

1,05 


Religion 

4,39 

,94 


Total 

4,46 

1,01 


Sport Sciences 

4,50 

1,H 


Human Sciences 

4,78 

,94 

Total 

Econ. and Manag. 

4,79 

1,12 

Science 

4,96 

1,07 


Religion 

4,94 

,87 


Total 

4,75 

1,04 


In the virtual application of the Information Literacy Scale, the highest average point was found for Humanities and 
Social Sciences Faculty (X = 5.38 ± .89) and the lowest average point was found for Sports Sciences Faculty (X = 4.73 
± 1.16) moreover, regarding printed application of the scale, the highest average point was found for Humanities and 
Social Sciences Faculty (4.75 ± 0.93) and the lowest average point was found for Science Faculty (X = 4.12 ± 1.05). 
When a general evaluation was made, it was determined that the average of the virtual survey application (X = 4.99 ± 
1.01) was higher than the average of the printed application (X = 4.46 ± 1.01). 

Within the content of the study, reliabilities of the printed and virtual scales were also investigated as it is important to 
test the findings in terms of reliability. Findings on reliability analysis are given in Table 8. 
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Table 8. Reliability Coefficients of the Scales in terms of Applied Methods 


Application 

MCTS 

ILSE 

ATTPS 

Smartphone 

,807 

,938 

,907 

Printed 

,883 

,924 

,922 


As it is seen in the table, reliability coefficients of printed applications of Metacognitive Thinking and Attitudes towards 
Teaching Profession scales, and virtual application of Information Literacy Scale are higher. Overall, reliability values 
are sufficient for all three scales and for two types of applications. 

5. Conclusion and Discussion 

The results of the current research show that no significant differences were obtained among attitude scores towards 
teaching profession with regard to common effect of application method and faculty variables. However, significant 
differences in participant opinions were determined among different application methods (virtual and printed) of 
Metacognitive Thinking Skills and Information Literacy Scales. In addition, no significant difference was found among 
opinions towards Metacognitive Thinking Skills and Information Literacy Scales with regard to common effect of 
application method and faculty variables. Besides, when general scale averages were evaluated, it was seen that virtual 
survey application score is lower than printed application score of Attitudes towards Teaching Profession Scale and 
printed survey application scores are lower than virtual application scores of Metacognitive Thinking Skills and 
Information Literacy Skills Scales. 

When an assessment is made in terms of scales, it has been determined that the average scores of printed survey of 
Attitudes towards Teaching Profession Scale is lower than its virtual version for Sport Sciences and Economics and 
Administrative Sciences faculties while, average scores of printed survey is higher than its virtual version for Science, 
Theology and Humanities and Social Sciences faculties. These findings obtained with this scale have not allowed 
making a realistic assessment. However, when an evaluation is made in terms of the mean scores of Metacognitive 
Thinking Skills Scale with regard to application method, it is seen that the average scores of the virtual application are 
higher for all faculties. The lowest difference among the scores regarding both application methods was obtained at the 
Faculty of Theology. When Information Literacy Scale averages were evaluated, it was determined that the virtual 
scores are higher in all faculties and the closest values of virtual and printed surveys were obtained for Humanities and 
Social Sciences Faculty. When all of these findings were evaluated together, a similarity (no significant difference in 
participant opinions) regarding Attitude towards Teaching Profession Scale, which is a common interest for the whole 
group in terms of item expressions, and higher points of virtual applications of Cognitive Thinking Skills and 
Information Literacy scales, which consist of more complex item expressions are attracting attention were seen. Another 
finding supporting this situation is that the average scores of virtual and printed applications for Humanities and Social 
Sciences Faculty, which contains departments such as history and literature and which its students are closely related to 
information literacy skills, are somewhat closer compared to other faculties. Accordingly, it can be said that the 
applications of virtual and printed surveys may differ significantly from each other and for this reason, may be 
suspicious in terms of validity of obtained data. 

When the findings were examined in terms of faculties and their departments, it will be necessary to explain significant 
differentiation of scores among applications. According to the impression obtained by the researcher, one reason of 
difference among the applications is the level of understanding and inteipretation. While the students who conducted 
the printed questionnaires were more deliberate about their own self-efficacy perceptions, the students who conducted 
the virtual survey application saw themselves more adequate on these topics. In the literature, several studies with 
similar findings were reported. It is thought that problems arising from screen reading may have been experienced when 
obtained findings and studies in the literature are evaluated together. 

Screen reading is defined as reading either half or the quarter of text on screen as divided pages (Sun, 2009: 317). 
Aysever (2004) argues that reading electronic texts on computer screen creates a feeling such as entering from one room 
to another room with more than one door. Giine§ (2010) and Altun and (^akmak (2008) are cautious about screen 
reading for some reasons such as disappearance of one part of a page while reading the other part, foreign characters 
complicating recognition of the words and nonexistence of initial and last pages thus, complicating perception of text in 
an ordered format. Whereas, reading is a complex process that has physiological, mental and spiritual aspects such as 
comprehending, analyzing and evaluating emotions in texts (Celik, 2006). According to one other researcher (Giinay, 
2004) reading, a text -regardless of its structure- where words are connected to each other in a meaningful way, is 
observing connections to discover the meaning derived from coexistence of words and sentences, connecting words to 
each other to interpret and to find another meaning other than lexical meaning. When all these opinions and findings are 
evaluated together, it is considered that the virtual survey group may not have understood enough the items they 
responded to. Perhaps, an experience effect is in question where virtual survey application yielded higher average scores. 
Filling in the questionnaires in different environments by a group of people who have been exposed more to printed 
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questionnaires in their lives has established a ground for this result. By all means, questions, such as whether filling in 
more virtual questionaries’ would have any effect or what effects would a customization create, may be described as 
critical. 

In an experimental study they performed, Tuncer and Bahadtr (2014) concluded that the printed-material group was 
more successful than the screen-reading group in terms of computer lesson achievement scores. In this study, the group 
(experiment and control) and the common effect of gender were investigated and no statistically significant difference 
was found. A further research on this research was done by Fleming and Bowden (2009). Fleming and Bowden (2009) 
compared mail and web-based survey data in terms of some demographic variables, and found no significant difference 
between the two methods. This result indicates that the findings may change as the environment changes. 

Reliability coefficients of Metacognitive Thinking Scale and Attitudes towards Teaching Profession Scale are higher for 
their printed applications whereas, reliability coefficient is higher for virtual application of Information Literacy Scale. 
This finding is similar to that of Morris, Woo and Cho (2003). Morris, Woo and Cho (2003) found that reliability of a 
measurement tool presented in an interactive medium is higher than its paper and pencil form and concluded that the 
results of non-cognitive tests are similar for both forms of application. On the other hand, in a research Szolnoki and 
Hoffman (2013) noted that face-to-face surveys gave the best results among the others where they implemented 
face-to-face, online and telephone survey methods and also that telephone surveys might be a good alternative (but 
larger sampling is needed) and a lot more corrections or some behavioral variables are needed to be taken into 
consideration for online surveys. Wiersme (2013) believes that image effects for online surveys can cause mediocre and 
minor problems on different devices, especially on personal computers and mobile phones. In addition, it has been 
noted that experiments conducted in one single environment via another means (such as entering the Internet by 
telephone) did not yield good results, and yet no favorable environments for smartphone-based questionnaires were 
formed. Nulty (2008) opposes to research that suggests more people can be reached by online surveys (Strachota, 
Schmidt and Conceicao, 2005; Schmidt, Strachota and Conceifao, 2006; Rose and Bogue, 2006) and states that 
responses to printed questionnaires are higher, but response rates can be increased by various methods. 

As the conclusion, based on these research findings, it is not recommended to distribute survey items containing 
technical information or some complex expressions via smartphones. Relations among the questionnaire content and the 
department of education confirm this situation. The reason of why the virtual survey group displayed a higher 
self-efficacy level is a topic that needs to be investigated. 
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